Skip to content

operations table: Store the operation_xdr as BYTEA#497

Draft
aditya1702 wants to merge 23 commits intohash-byteafrom
opxdr-bytea-2
Draft

operations table: Store the operation_xdr as BYTEA#497
aditya1702 wants to merge 23 commits intohash-byteafrom
opxdr-bytea-2

Conversation

@aditya1702
Copy link
Contributor

What

[TODO: Short statement about what is changing.]

Why

[TODO: Why this change is being made. Include any context required to understand the why.]

Known limitations

[TODO or N/A]

Issue that this PR addresses

[TODO: Attach the link to the GitHub issue or task. Include the priority of the task here in addition to the link.]

Checklist

PR Structure

  • It is not possible to break this PR down into smaller PRs.
  • This PR does not mix refactoring changes with feature changes.
  • This PR's title starts with name of package that is most changed in the PR, or all if the changes are broad or impact many packages.

Thoroughness

  • This PR adds tests for the new functionality or fixes.
  • All updated queries have been tested (refer to this check if the data set returned by the updated query is expected to be same as the original one).

Release

  • This is not a breaking change.
  • This is ready to be tested in development.
  • The new functionality is gated with a feature flag if this is not ready for production.

Part of the plan to store XDR data as raw bytes for efficiency.
Similar pattern to HashBytea but uses base64 encoding (standard for XDR)
and supports variable-length data.
Changed OperationXDR field from string to XDRBytea for automatic
base64/BYTEA conversion.
Cast the base64 XDR string to XDRBytea for proper database storage.
- Change UNNEST cast from text[] to bytea[]
- Convert XDRBytea to raw bytes in BatchInsert
- Update BatchCopy to pass raw bytes instead of pgtype.Text
Forces GraphQL to use a resolver for operationXdr to convert
XDRBytea to base64 string for API consumers.
- Update test_utils.go to use types.XDRBytea
- Regenerate GraphQL code with OperationXdr resolver
Returns the XDRBytea as a base64-encoded string for API consumers.
- Use parameterized queries with XDRBytea for SQL inserts
- Update assertions to use .String() for comparison
- Fix type casting in test data creation
- Use base64-encoded test XDR data in test_utils.go
- Add testOpXDR helper functions in test files
- Update all assertions to use .String() method
- Update createTestOperation to use proper base64 XDR
- Update generateTestOperations to encode test data as base64
- Update operations_test.go with base64 encoding for all test XDR data
- Fix assertions to compare against the stored XDRBytea values
This simplifies the XDR storage flow by storing raw bytes directly
instead of encoding to base64 and then decoding. The String() method
now handles base64 encoding for external representation.
Skip the intermediate base64 encoding step by using MarshalBinary()
instead of MarshalBase64(). The raw bytes are now stored directly
in XDRBytea.
Remove unnecessary Value() calls since XDRBytea is now []byte.
Access raw bytes directly via type conversion.
Decode expected base64 XDR string to raw bytes for comparison
since XDRBytea now uses []byte underlying type.
Use raw bytes directly instead of base64-encoded strings when
creating test data for XDRBytea fields.
Use raw bytes directly for test XDR data instead of base64-encoding.
The String() method will handle base64 encoding for assertions.
Use raw bytes directly instead of pre-encoded base64 string.
Use parameterized queries instead of raw SQL string literals
for BYTEA operation_xdr column. Fix .String() assertion to
compare base64 values via opXdr1.String().
Use parameterized queries instead of raw SQL string literals
for BYTEA operation_xdr column in BatchGetByOperationIDs and
BatchGetByStateChangeIDs tests.
Use parameterized queries instead of raw SQL string literals
for BYTEA operation_xdr column in BatchGetByOperationIDs test.
Copy the byte slice from the database driver instead of
referencing it directly. The pgx driver reuses its internal
buffer across rows, so without copying, all scanned XDRBytea
values end up pointing to the same (overwritten) buffer.
@aditya1702 aditya1702 changed the title Opxdr bytea 2 operations table: Store the operation_xdr as BYTEA Feb 5, 2026
@aditya1702 aditya1702 requested a review from Copilot February 11, 2026 13:54
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the operations.operation_xdr storage to use PostgreSQL BYTEA (raw XDR bytes) while preserving the GraphQL API contract by continuing to expose operationXdr as a base64-encoded String.

Changes:

  • Change operations.operation_xdr from TEXT to BYTEA in the schema/migration and update DB write paths accordingly.
  • Introduce types.XDRBytea to scan/store raw bytes and encode to base64 when presented as a string.
  • Update GraphQL schema/resolvers and tests to reflect the new storage/encoding behavior.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/services/ingest_test.go Updates ingest test fixtures to use types.XDRBytea.
internal/serve/graphql/schema/operation.graphqls Forces resolver for operationXdr so GraphQL continues returning a base64 string.
internal/serve/graphql/resolvers/transaction_resolvers_test.go Updates expectations for base64-encoded XDR output.
internal/serve/graphql/resolvers/test_utils.go Stores operation XDR as raw bytes in test DB setup.
internal/serve/graphql/resolvers/queries_resolvers_test.go Updates query resolver tests to compare base64 encoding of raw bytes.
internal/serve/graphql/resolvers/operation.resolvers.go Adds OperationXdr field resolver to base64-encode XDRBytea.
internal/serve/graphql/resolvers/account_resolvers_test.go Updates account resolver tests for base64-encoded XDR output.
internal/serve/graphql/generated/generated.go Regenerates gqlgen output to wire the forced resolver.
internal/indexer/types/types.go Adds XDRBytea type and updates types.Operation.OperationXDR to use it.
internal/indexer/processors/utils_test.go Updates ConvertOperation test to compare raw bytes vs base64 string.
internal/indexer/processors/utils.go Stores raw XDR bytes via MarshalBinary() instead of base64 strings.
internal/db/migrations/2025-06-10.3-operations.sql Changes operation_xdr column type to BYTEA (but see migration concern).
internal/data/transactions_test.go Updates test inserts to pass XDRBytea for operation_xdr.
internal/data/operations_test.go Updates operation model tests for bytea storage and base64 string comparisons.
internal/data/operations.go Updates batch insert/copy paths to send operation_xdr as bytea.
internal/data/accounts_test.go Updates account model tests inserting operations to use XDRBytea.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 19 to 21
),
operation_xdr TEXT,
operation_xdr BYTEA,
result_code TEXT NOT NULL,
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the column type in this existing migration file won’t update already-migrated databases, so production/dev DBs that have operation_xdr as TEXT will keep that type and will likely break once the app starts inserting BYTEA. Add a new forward migration that ALTER TABLE operations ALTER COLUMN operation_xdr TYPE BYTEA USING decode(operation_xdr, 'base64') (or equivalent), and avoid rewriting already-shipped migrations.

Copilot uses AI. Check for mistakes.
id: Int64!
operationType: OperationType!
operationXdr: String!
operationXdr: String! @goField(forceResolver: true)
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description still contains TODO placeholders (What/Why/Issue/Checklist). Please fill these in so reviewers understand the rationale, rollout/migration plan, and any breaking-change considerations for the operation_xdr storage change.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant